Prompting Is Programming: A Query Language for Large Language Models

Large language models have demonstrated outstanding performance on a wide range of tasks such as question answering and code generation. On high level, given an input, model can be used to automatically complete the sequence in statistically-likely way. Based this, users prompt these with instructions or examples, implement variety downstream tasks. Advanced prompting methods even imply interaction between model, user, external tools calculators. However, obtain state-of-the-art adapt for specific tasks, complex task- model-specific programs implemented, which may still require ad-hoc interaction. we present novel idea Language Model Programming (LMP). LMP generalizes from pure text prompts intuitive combination scripting. Additionally, allows constraints specified over output. This enables easy adaption many while abstracting internals providing high-level semantics. To enable LMP, LMQL(short Query Language), leverages control flow generate efficient inference procedure that minimizes number expensive calls underlying model. We show LMQL capture way, especially facilitating interactive flows are challenging existing APIs. Our evaluation shows retain increase accuracy several also significantly reducing required amount computation cost case pay-to-use APIs (26-85% savings).